System And Method Of Material Handling Using One Imaging Device On The Receiving Vehicle To Control The Material Distribution Into The Storage Portion Of The Receiving Vehicle

ABSTRACT

A single imaging device collects image data of a storage portion. A container module identifies a container perimeter of the storage portion in at least one of the collected image data. A spout module is adapted to identify a spout of the transferring vehicle in the collected image data. An arbiter determines whether to use the image data based on an evaluation of the intensity of pixel data or ambient light conditions. An alignment module is adapted to determine the relative position of the spout and the container perimeter and to generate command data to the propelled portion to steer the storage portion in cooperative alignment such that the spout is aligned within a central or target zone of the container perimeter.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation In Part application of PCTInternational Application PCT/US2013/025572, filed Feb. 11, 2013, titledSYSTEM AND METHOD OF MATERIAL HANDLING USING ONE IMAGING DEVICE ON THERECEIVING VEHICLE TO CONTROL THE MATERIAL DISTRIBUTION INTO THE STORAGEPORTION OF THE RECEIVING VEHICLE, which claims the priority of U.S.Provisional Application 61/597,346, filed Feb. 10, 2012, and U.S.Provisional Application 61/597,374, filed Feb. 10, 2012, and U.S.Provisional Application 61/597,380, filed Feb. 10, 2012, all areincorporated by reference herein.

JOINT RESEARCH AGREEMENT

This application resulted from work performed under or related to ajoint research agreement between Carnegie Mellon University and Deere &Company, entitled “Development Agreement between Deere & Company andCarnegie Mellon University,” dated Jan. 1, 2008 and as such is entitledto the benefits available under 35 U.S.C. §103(c).

FIELD OF THE INVENTION

This invention relates to a method and stereo vision system forfacilitating the unloading of material from a vehicle.

BACKGROUND

Certain prior art systems may attempt to use global positioning system(GPS) receivers to maintain proper spacing between two vehicles duringthe unloading or transferring of agricultural material or othermaterials, such as coal or other minerals, between the vehicles.However, such prior art systems are susceptible to misalignment of theproper spacing because of errors or discontinuities in the estimatedposition of the GPS receivers. For example, one or more of the GPSreceivers may misestimate its position because of electromagneticinterference, multipath propagation of the received satellite signals,intermittent reception of the satellite signals or low received signalstrength of the satellite signals, among other things. If the vehiclesuse cameras or other imaging devices in an outdoor work area, such as anagricultural field, the imaging devices may be subject to transitorysunlight, shading, dust, reflections or other lighting conditions thatcan temporarily disrupt proper operation of the imaging devices; hence,potentially produce errors in estimated ranges to objects observed bythe imaging devices. Thus, there is a need for an improved system formanaging the unloading of agricultural material from a vehicle tocompensate for or address error in the estimated positions or alignmentof the vehicles.

SUMMARY OF THE INVENTION

The system and method facilitates the transfer of agricultural materialfrom a transferring vehicle (e.g., harvesting vehicle) to a receivingvehicle (e.g., grain cart). The system and method comprises a receivingvehicle, which has a propelled portion for propelling the receivingvehicle and a storage portion for storing agricultural material and atransferring vehicle for transferring harvested agricultural materialinto the storage portion of the receiving vehicle.

Two embodiments of the present invention include one or two primaryimaging devices on only one vehicle, either the receiving vehicle or thetransferring vehicle. A first embodiment mounts only one primary imagingdevice on the propelled portion of the receiving vehicle and no imagingdevices mounted on the transferring vehicle. A second embodiment mountsone or two imaging devices on the transferring vehicle and no imagingdevices on the receiving vehicle.

The receiving vehicle and/or the transferring vehicle of any of theabove mentioned embodiments can include as an image processing modulehaving a container module that can identify a container perimeter of thestorage portion in at least one of the collected first image data andthe collected second image data (where a second imaging device isincorporated into the system configuration). The image processing canalso include a spout module that is adapted to identify a spout of thetransferring vehicle in the collected image data (collected first imagedata, collected second image data, or both). The image processing modulecan include an arbiter (e.g., image data evaluator) that determineswhether to use the first image data, the second image data or both(where a second imaging device is incorporated into the systemconfiguration), based on an evaluation of material variation ofintensity of pixel data or material variation in ambient lightconditions during a sampling time interval. In a system with only oneimaging device, the arbiter either not activated, is not incorporatedinto the system, or includes logic that passes the only collect image tothe next function. The image processing module can also include analignment module that is adapted to determine the relative position ofthe spout and the container perimeter, and to generate command data tothe propulsion controller of the transferring vehicle or the receivingvehicle or both to propel (accelerate or decelerate) the storage portionin cooperative alignment with the transferring vehicle such that thespout is aligned within a central zone (or other target zone) of thecontainer perimeter. The present invention of any of the embodiment caninclude a steering controller that is associated with a steering systemof the transferring vehicle or the receiving vehicle or both forsteering the receiving vehicle in accordance with the cooperativealignment with the transferring vehicle based on input from alignmentmodule. The transferring vehicle can include a material profile moduleto develop a profile of the material within the storage portion of thereceiving vehicle to facilitate vehicle cooperative alignment and spoutadjustment.

In operation, a method for facilitating the transfer of material from atransferring vehicle having a material distribution end to a receivingvehicle having a bin to the store transferred material, the methodcomprising the steps of:

a. identifying and locating the bin;

b. detecting a representation of the fill level or volumetricdistribution of the material in the bin;

c. aligning the material distribution end over a current target area ofthe bin requiring the material (wherein a current target area can be aninitial target area the material distribution end is positioned when thefilling of material begins);

d. determining subsequent target areas of the bin that require materialbased on the representation of the fill level or volumetric distributionof the material in the bin and a desired fill pattern (such asfront-to-back, back-to-front, center-to-front-to-back,center-to-back-to-front) to fill the bin;

e. transferring the material from the transferring vehicle to thecurrent target area of the bin of the receiving vehicle;

f. detecting when the current target area of the bin is filled with thematerial; and

g. repeating steps c-f until the subsequent target areas of the bin arefilled per the desired fill pattern.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of one embodiment of a machinevision-augmented guidance system for a transferring vehicle forfacilitating the unloading of agricultural material from thetransferring vehicle (e.g., combine);

FIG. 2 is a block diagram of another embodiment of a machinevision-augmented guidance for a transferring vehicle for facilitatingthe unloading of agricultural material from the transferring vehicle(e.g., a self-propelled forage harvester);

FIGS. 3A and 3B are block diagrams of embodiments of a machinevision-augmented guidance system for a receiving vehicle forfacilitating the unloading of agricultural material from a transferringvehicle to the receiving vehicle (e.g., grain cart and tractor);

FIG. 4A illustrates a top view of an imaging devices mounted on atransferring vehicle and facing toward a receiving vehicle;

FIG. 4B illustrates a view in a horizontal plane as viewed alongreference line 4B-4B in FIG. 4A;

FIG. 5A illustrates a top view of a single imaging device (e.g., astereo vision system) mounted on a receiving vehicle and facing astorage portion of the receiving vehicle;

FIG. 5B illustrates a view in a horizontal plane as viewed alongreference line 5B-5B in FIG. 5A;

FIG. 5C illustrates a two-dimensional representation of various possibleillustrative distributions of material in the interior of a container orstorage portion, consistent with a cross-sectional view along referenceline 5D-5D in FIG. 5B;

FIG. 5D is a plan view of a transferring vehicle and a receivingvehicle, where the transferring vehicle is aligned within a matrix ofpossible offset positions;

FIG. 6 illustrates a block diagram of a container module or an imageprocessing module;

FIG. 7 is a block diagram of a spout module or an image processingmodule;

FIG. 8 is a flow chart of a method for operating a mode controller of amachine vision-augmented guidance system for facilitating the unloadingof agricultural material from a vehicle (e.g., combine);

FIG. 9 is a flow chart of yet another method for a machinevision-augmented guidance system for facilitating the unloading ofagricultural material from a vehicle (e.g., combine);

FIG. 10 is a flow chart of another method for a machine vision-augmentedguidance system for facilitating the unloading of agricultural materialfrom a vehicle (e.g., combine); and

FIG. 11 is a schematic illustrating the data flow and processing by theimage processing module from raw images to vehicle commands.

DETAILED DESCRIPTION OF THE INVENTION

In accordance with one embodiment of the present invention that requiresimaging devices only on the receiving vehicle 79, FIGS. 3A, 5A and 5Bshow a machine vision augmented guidance system 311 for a receivingvehicle 79 for managing the unloading of agricultural material (e.g.,grain) from the transferring vehicle 91 (e.g., combine) to a receivingvehicle 79 (e.g., grain cart or wagon). FIG. 5A illustrates a top viewof an exemplary transferring vehicle 91 and a receiving vehicle 79configuration. FIG. 5B illustrates a side view of an exemplarytransferring vehicle 91 and a receiving vehicle 79 configuration of FIG.5A. For example, a stereo imaging system augments satellite navigationreceivers or location-determining receivers 142 for guidance ofreceiving vehicle 79. The imaging device 10 has a field of view 277,indicated by the dashed lines. The boundaries of the field of view 277are merely shown for illustrative purposes and will vary in actualpractice. The system 311 can comprise a imaging device 10 coupled to animage processing module 18. Embodiments of imaging device 10 maycomprise a stereo camera. Like element numbers provided for herein havethe same function or meaning. Though the example of transferred materialdisclosed herein is agricultural material, the invention is not to belimited to agricultural material and is applicable to other materialssuch as coal and other minerals.

Now turning to FIG. 11 that illustrates the data flow and processing bythe image processing module 18 from the raw images to the receivingvehicle commands. The components and modules will be discussed in detailbelow. The dashed lines represent optional steps and/or modules. Rawimages are collected by the imaging device 10 (e.g. camera beingstereo). Raw images are processed through the image rectifier 101 tocreate rectified images. Rectified images are processed by the arbiteror image data evaluator 25 to provide an image quality score for therectified image to determine if the image should be used in furtherprocessing by the alignment module 24. Rectified images are alsoprocessed by the container (identification) module 20 and materialprofile module 27. Rectified images can be processed by a disparitygenerator 103 to generate range data with regards to the container orbin 85 characteristics, such as distances from and dimensions of edges,length, and depth. Rectified images can also be processed in conjunctionwith disparity images by the spout localizer or module 22 when adisparity image generator 103 is present. Otherwise, spout localizer 22will only use the data stored in vehicle model 1000, which includes, butnot limited to, data on the transferring vehicle 91, dimensions of spout89, spout kinematic model and container information. Spout localizer 22also requires data about the vehicle state information, which includes,but not limited to, transferring vehicle speed, spout angle(s), augerdrive on/off status, and relative Global Positioning Satellite positionof receiving vehicle 79 if machine synchronization is present. Spoutlocalizer 22 output (e.g. spout orientation) is input into containeridentification module 20 and processed in conjunction with rectifiedimages and disparity images (if provided) by container identificationmodule 20 and container dimensions provided by the vehicle model 1000 todetermine container location and dimensions. Rectified images anddisparity images (if provided) are processed in conjunction withcontainer location and dimensions data from container identificationmodule 20 by material profile module 27 to generate a fill profile ofthe container 85. Alignment module 24 processes data generated by thecontainer identification module 20, material profile module 27, vehicledimensions provided by the vehicle model 1000 in conjunction with thevehicle state information to generate vehicle commands such as receivingvehicle 79 speed/steering, spout position, auger drive on/off status,and speed/steering of the receiving vehicle 79 if machinesynchronization is present to reposition the spout end 87 over theappropriate open area of the container 85 for even, uniform distributionof the agricultural material in container 85.

Now returning to FIG. 3A that comprises the first imaging device 10 theimage processing module 18, the user interface processing module 26, thegateway 29, a second location-determining receiver 142, a secondwireless communications device 148, the slave controller 159 among otherdevices illustrated in FIG. 3A. In one embodiment, the first imagingdevice 10 is mounted on the propelled portion 75 (e.g., tractor) of thereceiving vehicle 79 facing backwards towards the storage portion 93(e.g., cart) or container 85. The second wireless communications device148 of the receiving vehicle 79 is adapted for communicating data withthe first communications device 48 of the transferring vehicle 91 ofFIG. 4A. The second location-determining receiver 142 provides positiondata, location data, altitude, velocity, or acceleration data.

The slave controller 159 can operate in a slave mode or follower modeunder the control of the master controller 59. The auto-guidance module155 and the coordination module 157 within the slave controller 159provide guidance of the receiving vehicle 79, consistent with locationdata and a path plan, or with other guidance data or command data fromthe transferring vehicle 91.

The second wireless communications device 148 is coupled to the vehicledata bus 60. In FIG. 3A, the system 311 for a receiving vehicle can beused in conjunction with the system 11 or 111 of the transferringvehicle 91 of FIG. 1 or, or independently of any transferring vehicle.The wireless devices 48, 148 may exchange or communicate position data,relative position data, command data, or control data for controlling,adjusting or coordinating the position and orientation of the vehicles;more particularly, the position and the orientation of the spout 89 orspout end 87 over the opening 83 of the container 85. The communicateddata between the wireless communications devices 48, 148 may compriseany of the following data: (1) position data or location data fromeither location determining receiver 42 or 142, (2) command or guidancedata from an image processing module 18 on the transferring vehicle 91or receiving vehicle 79, (3) command or guidance data from the mastercontroller 59 or coordination module 47, (4) command or guidance datafrom the slave controller 159 or coordination module 157 or (5)alignment data (e.g., relative position of the imaging devices, relativeposition of reference points on the vehicles, and relative alignmentbetween the spout 89 and container perimeter 81) from the alignmentmodule 24. For example, the imaging processing module 18 or alignmentmodule 24 may use first location data of a first location determiningreceiver 42 and second location data of a second location determiningreceiver 142 to determine a relative position or spatial offset betweenthe two vehicles (or a relative position) of the first imaging device 10and the second imaging device 12 and to determine a relative alignmentbetween the spout 89 and the container perimeter 81.

The system 311 of FIG. 3A may support different configurations orcombinations of electronic systems (e.g., 11 and 311 or 111 and 311) atthe transferring vehicle 91 and receiving vehicle 91. In a firstconfiguration, only one imaging device 10 is on the receiving vehicle 79may be used instead of, or with, one or more imaging devices 10, 12 onthe transferring vehicle 91. In a second configuration, the system 311of FIG. 3A may provide collected image processing data from thereceiving vehicle 79 to the transferring vehicle 91 via the transmissionof the collected image processing data from the second wirelesscommunications device 148 to the first wireless communications device48. Here, in a second configuration, the collected imaging processingdata from the receiving vehicle 79 may be referred to as supplementarydata, complementary image data, or additional image data. The additionalimage data may provide additional perspective or viewpoints that cansupplement the image data collected by the transferring vehicle 91. Forexample, the additional image data may provide more accurate orsupplement image data where the image data collected by the transferringvehicle 91 is affected by moisture (e.g., on its lens), dust, poorambient lighting, glare or reflections that do not similarly impair orimpact the additional image data.

The optional odometry sensor 440 may be coupled to the vehicle data bus60 or the implement data bus 58. The inertial sensor 442 may compriseone or more accelerometers, gyroscopes or other inertial devices coupledto the vehicle data bus 31 or the implement data bus 60.

The distributed fill state sensors 149 (FIG. 3A) may comprise opticallevel sensors (not shown) distributed at different height levels withinor around the container 85, piezoelectric mass sensors distributed tomeasure mass of the agricultural material in different volumes or ondifferent floor areas (e.g., of a false vertically movable floor) of thecontainer 85, or piezoresistive mass sensors distributed to measure massof the agricultural material in different volumes or on different floorareas of the container 85, for example.

FIG. 5A illustrates a top view of a transferring vehicle 91 and areceiving vehicle 79. Like reference numbers indicate like elements inFIG. 5A and FIG. 4A. FIG. 5A shows an imaging device 10 on the rear ofthe propulsion unit 75 (e.g., tractor) or the receiving vehicle 79. Theimaging device 10 has a field of view 277 indicated by the dashed lines.In FIG. 5A, the spout 89 or spout end 87 is generally aligned over acentral zone 83, central region or target area of the storage unit 93 orcontainer 85 for unloading material from the transferring vehicle 91 tothe receiving vehicle 79. Similarly, the transferring vehicle 91 and thereceiving vehicle 79 are aligned in position as shown, and even as thevehicles 79, 91 move with coordinated headings or generally parallelheadings and with no or minimal relative velocity with respect to eachother. In FIG. 5A, the image processing module 18 can estimate thedistance or range from the imaging device 10 to an object in the image,such as the spout 89, the spout end 87, the container perimeter 81, thelevel or profile of agricultural material in the container 85 (e.g., atvarious positions or coordinates within the container 85). The term“bin” can be used in place of the term “container.”

FIG. 5B illustrates a view in a horizontal plane as viewed alongreference line 5B-5B in FIG. 5A. In one embodiment, the first imagingdevice 10 is mounted on the receiving vehicle 79 on a first support 571(e.g., monopole with tilt or pan adjustment) to provide a first downwardfield of view 577 or a first down-tilted field of view.

In an alternate embodiment, the first support 571 comprises anadjustable mast or telescopic mast that is controlled by a mastcontroller 674 to remotely adjust the height, tilt angle, down-tiltangle, rotation angle, or pan angle to provide reliable image data forprocessing by the image processing module 18. Similarly, the secondsupport 573 comprises an adjustable mast or telescopic mast that iscontrolled by a mast controller (674) to remotely adjust the height,tilt angle, down-tilt angle, rotation angle, or pan angle to providereliable image data for processing by the image processing module 18.

FIG. 5C illustrates a two-dimensional representation of various possibleillustrative distributions of material in the container 85, consistentwith a view along reference line 5C in FIG. 5A. In one configuration,the y axis is coincident with the longitudinal axis or direction oftravel of the container, the z axis is coincident with the height ofmaterial in the container, and the x axis is perpendicular to thedirection of travel of the container, where the x, y and z axes aregenerally mutually orthogonal to each other.

In the chart of FIG. 5C, the vertical axis is the mean height (f) 500 ofthe material in the container 85, the horizontal axis represents thelongitudinal axis (y) 502 of the container 85. The maximum capacity 504or container capacity is indicated by the dashed line on the verticalaxis. The front 512 of the container 85 is located at the origin,whereas the back 514 of the container 85 is located on the verticalaxis.

FIG. 5C shows three illustrative distributions of material within thecontainer 85. The first distribution is a bimodal profile 508 in whichthere are two main peaks in the distribution of material in thecontainer 85. The bimodal profile 508 is shown as a dotted line. Thebimodal profile 508 can occur where the spout angle adjustment isgoverned by an electro-hydraulic system with non-proportional valves.

The second distribution is the front-skewed modal profile 510 in whichthere is single peak of material toward the front of the container 85.The front-skewed modal profile 510 is shown as alternating long andshort dashes. The second distribution may occur where the volume orlength (y) of the container 85 is greater than a minimum threshold andwhere the relative alignment between the spout end 87 and the container85 is generally stationary during a substantial portion of unloading ofthe material.

The third distribution is the target profile 508 which may be achievedby following a suitable fill strategy as disclosed in this document. Forexample, during unloading, the spout angle may be adjusted to promoteuniform distribution of the agricultural material in the container 85.Further, the lateral offset (Δ) or fore/aft offset (Φ or φ) between thevehicles 79, 91 may be adjusted in accordance with a matrix (e.g., x, ycoordinate matrix of equidistant point locations of the transferringvehicle relative to a constantly spaced position point of the receivingvehicle) of relative unloading positions, particularly for longer orwider containers that cannot be uniformly filled from a single, relativeunloading point between the vehicles 79, 91.

FIG. 5D is a plan view of a transferring vehicle 91 and a receivingvehicle 79, where the transferring vehicle 91 is aligned within a matrix500 of possible offset positions 502, 504 between the transferring andreceiving vehicle 79. Each offset position 502, 504 may be defined interms of a combination of a unique lateral offset (Δ) and a uniquefore/aft offset (Φ or φ) between the vehicles 79, 91. As shown, thematrix 500 is a two-dimensional, 2×3 (2 columns by 3 rows) matrix ofpossible offset positions 502, 504. Although six possible matrixpositions 502, 504 are shown, in alternate embodiments the matrix 500may consistent of any number of possible offset positions greater thanor equal to two. Here, the transferring vehicle 91 occupies a currentoffset position 504 in the first column at the second row of the matrix500, whereas the other possible offset positions 502 are not occupied bythe transferring vehicle 91. As directed by any of the systems (11, 111,311), the imaging processing module 18, or the master controller 59 ofthe transferring vehicle 91 can shift to any unoccupied or otherpossible offset positions 502 within the matrix 500 to promote orfacilitate an even distribution of agricultural material within thecontainer 85 or storage portion of the receiving vehicle 79. The spatialoffset between the transferring vehicle 91 and the receiving vehicle 79may be adjusted in accordance with the matrix 500 or another matrix ofpreset positions of spatial offset to promote even distribution ofagricultural material in the storage portion of the receiving vehicle79, where any matrix is associated with a unique, relative lateraloffset (Δ) and fore/aft offset (Φ or φ) between the vehicles 79, 91.

In accordance with another embodiment of the present invention thatrequires imaging devices only on the transferring vehicles, FIGS. 1, 3B,4A and 4B show a machine vision augmented guidance system 11 for atransferring vehicle 91 for managing the unloading of agriculturalmaterial (e.g., grain) from the transferring vehicle 91 (e.g., combine)to a receiving vehicle 79 (e.g., grain cart or wagon), and FIGS. 2, 3B,4A and 4B show a machine vision augmented guidance system 111 for atransferring vehicle 91 for managing the unloading of agriculturalmaterial (e.g., grain) from the transferring vehicle 91 (e.g.,self-propelled forge harvester) to a receiving vehicle 79 (e.g., graincart or wagon). FIG. 4A illustrates a top view of an exemplarytransferring vehicle 91 and a receiving vehicle 79 configuration. Forexample, a stereo imaging system augments satellite navigation receiversor location-determining receivers 42 for guidance of one or morevehicles. The first imaging device 10 has a first field of view 77,indicated by the dashed lines. The second imaging device 12 has a secondfield of view 177, indicated by the dashed lines. The boundaries of thefields of view 77, 177 are merely shown for illustrative purposes andwill vary in actual practice. The system 11 can comprises a firstimaging device 10 and a second imaging device 12 coupled to an imageprocessing module 18. Embodiments of first imaging device 10 maycomprise a primary stereo camera or a monocular camera, while the secondimaging device 12 may comprise a secondary stereo camera or a monocularcamera. In one configuration, the second imaging device 12 is a stereocamera and can be optional and provides redundancy to the first imagingdevice 10 in case of failure, malfunction or unavailability of imagedata from the first imaging device 10 when the first field of view 77 ofthe first imaging device 10 is sufficient to view within container 85.In one configuration, the second imaging device is monocular and isrequired for a stereo image of the container 85 when used in conjunctionwith an image from a monocular first imaging device 10 with the firstfield of view 77 sufficient to view within container 85. Though theexample of transferred material disclosed herein is agriculturalmaterial, the invention is not to be limited to agricultural materialand is applicable to other materials such as coal and other minerals.FIG. 4A shows a first imaging device 10 on the transferring vehicle 91(e.g., combine) and a second imaging device 12 on a spout 89 of thetransferring vehicle 91. The second imaging device 12 can be optional ifthe first imaging device 10 is a stereo camera and the first field ofview 77 of the first imaging device 10 is sufficient to view withincontainer 85. The spout 89 may also be referred to as an unloadingauger. The spout end 87 may be referred to as a boot. In FIG. 4A, thespout 89, or the spout end 87, is generally aligned over a central zone83, central region or target area of the storage container 85 (of thereceiving vehicle 79) for unloading material from the transferringvehicle 91 to the receiving vehicle 79. Similarly, the transferringvehicle 91 and the receiving vehicle 79 are aligned in position asshown, regardless of whether the vehicles move together in a forwardmotion (e.g., with coordinated or tracked vehicle headings) duringharvesting, as is typical, or are stationary. During unloading, themaster controller 59 (FIGS. 1 and 2) and slave controller 159 (FIG. 3B)facilitate maintenance of a generally uniform spatial offset (e.g., agenerally static offset that varies only within a predetermined targettolerance) between the vehicles 91, 79, subject to any incrementaladjustment of the offset for uniform filling of the container 85. Themaster controller 59 and slave controller 159 support maintenance of auniform fore/aft offset (Φ) or (φ) and a lateral offset (Δ).

Now returning to FIG. 1, the transferring vehicle 91 may be equippedwith a rotation sensor 116 (e.g., rotary position sensor) to measure therotation angle of the spout. For a spout-mounted imaging device (e.g.,second imaging device 12 on the spout as shown in FIG. 4A), the rotationangle of the spout 89 may be used to facilitate fusion of image datafrom the first imaging device 10 and the second imaging device 12, or toconstruct stereo image data where the first imaging device 10 and thesecond imaging device 12 individually provide monocular image data forthe same scene or object.

In any arrangement of imaging devices 10, 12 disclosed herein where thefields of view 77, 177 overlap, data fusion of image data from a firstimaging device 10 and a second imaging device 12 enables the imageprocessing module 18 to create a virtual profile of the materialdistribution level inside the storage portion 85, even when the entiresurface of the agricultural material is not visible to one of the twoimaging devices 10, 12. Even if the second imaging device 12 is notmounted on the spout 89 in certain configurations, the rotation sensor116 may facilitate using the spout end 87 as a reference point in anycollected image data (e.g., for fusion, virtual stitching or alignmentof image data from different imaging devices.) The virtual profile ofthe entire surface of the agricultural material in the storage portion93 enables the system 11, 111, 411 or imaging module 18 to intelligentlyexecute a fill strategy for the storage portion 93 of the receivingvehicle.

The first imaging device 10 and the second imaging device 12 may providedigital data format output as stereo video image data or a series ofstereo still frame images at regular or periodic intervals, or at othersampling intervals. Each stereo image (e.g., the first image data or thesecond image data) has two component images of the same scene or aportion of the same scene. For example, the first imaging device 10 hasa first field of view 77 of the storage portion 93 of the receivingvehicle 79, where the first field of view 77 overlaps at least partiallywith a second field of view 177 of the second imaging device 12 (ifpresent).

In one configuration, an optical sensor 110, 112 comprises a lightmeter, a photo-sensor, photo-resistor, photo-sensitive device, acadmium-sulfide cell, charge-couple device, or complementary metal oxidesemi-conductor. A first optical sensor 110 may be associated with thefirst imaging device 10; a second optical sensor may be associated withthe second imaging device 12. The first optical sensor 110 and thesecond optical sensor 112 each may be coupled to the image processingmodule 18. The optical sensor 110, 112 provides a reading or levelindicative of the ambient light in the field of view of its respectiveimaging device 10, 12.

The image processing module 18 may be coupled, directly or indirectly,to lights 14 on a vehicle (e.g., transferring vehicle) for illuminationof a storage container 85 (FIG. 4A) and/or spout 89 (FIG. 4A). Forexample, the image processing module 18 may control drivers, relays orswitches, which in turn control the activation or deactivation of lights14 on the transferring vehicle. The image processing module 18 mayactivate the lights 14 on the vehicle for illumination of the storagecontainer 85 (FIG. 4A), spout 89 or both if an optical sensor 110, 112or light meter indicates that an ambient light level is below a certainminimum threshold. In one configuration the optical sensor 110, 112 facetoward the same direction as the lens or aperture of the imaging devices10, 12.

In one embodiment, the auger rotation system 16 of FIG. 1 may comprise:(1) a rotation sensor 116 for sensing a spout rotation angle (α in FIG.4A and β in FIG. 4C) of the spout 89 with respect to one or more axes ofrotation and (2) an actuator 216 for moving the spout 89 to change thespout rotation angle; hence, the spout position with respect to thereceiving vehicle 79 or its storage container 85. The rotation actuator210 may comprise a motor, a linear motor, an electro-hydraulic device, aratcheting or cable-actuated mechanical device, or another device formoving the spout 89, or the spout end 87. The spout rotation angle maycomprise a simple angle, a compound angle or multi-dimensional anglesthat is measured with reference to a reference axis parallel to thedirection of travel of the transferring vehicle.

If the rotation actuator 210 comprises an electro-hydraulic device, theuse of proportional control valves in the hydraulic cylinder of theelectro-hydraulic device that rotates the spout (or changes the spoutrotation angle) facilitates finer adjustments to the spout angle (e.g.,a) than otherwise possible. Accordingly, proportional control valves ofthe electro-hydraulic device support or rotation actuator 201 an evenprofile or distribution of unloaded agricultural material within thestorage portion 93 or container 85. Many commercially available combinesare typically equipped with non-proportional control valves forcontrolling spout angle or movement of the spout 89; electro-hydraulicdevices with non-proportional control valves can fill the storagecontainer with an inefficient multi-modal or humped distribution (e.g.,508) of agricultural material with local high areas and local low areas,as depicted in FIG. 5C, for example.

A vehicle controller 46 of FIG. 1 may be coupled to the vehicle data bus60 to provide a data message that indicates when the auger drive 47 forunloading agricultural material from the transferring vehicle isactivate and inactive. The auger drive 47 may comprise an auger, anelectric motor for driving the auger, and a rotation sensor for sensingrotation or rotation rate of the auger or its associated shaft. In oneembodiment, the auger (not shown) is associated with a container forstoring agricultural material (e.g., a grain tank) of a transferringvehicle 91 (e.g., a combine). If the vehicle controller 46 (e.g., augercontroller) indicates that the auger of the transferring vehicle isrotating or active, the imaging processing module 18 activates the spoutmodule 22 and container module 20. Thus, the auger rotation system 16may conserve data processing resources or energy consumption by placingthe container module 20 and the spout module 22 in an inactive state (orstandby mode) while the transferring vehicle is harvesting, but notunloading, the agricultural material to the receiving vehicle.

Spout controller 54 of FIG. 2 may be coupled to the vehicle data bus 60to provide a data message that indicates when spout will discharge ordistribute material based on data from the rotation actuator 260, tiltactuator 261, and deflector actuator 264.

In FIGS. 1 and 2, the imaging processing module 18 or any othercontroller may comprise a controller, a microcomputer, a microprocessor,a microcontroller, an application specific integrated circuit, aprogrammable logic array, a logic device, an arithmetic logic unit, adigital signal processor, or another data processor and supportingelectronic hardware and software. As mentioned above, in one embodimentthe image processing module 18 comprises a container module 20, a spoutmodule 22, an alignment module 24, a material profile module 27, and anarbiter 25.

The image processing module 18 may be associated with a data storagedevice 19. The data storage device 19 may comprise electronic memory,non-volatile random access memory, a magnetic disc drive, an opticaldisc drive, a magnetic storage device or an optical storage device, forexample. If the container module 20, the spout module 22 and thealignment module 24, material profile module 27 and arbiter 25, aresoftware modules they are stored within the data storage device 19.

The container module 20 identifies a set of two-dimensional or threedimensional points (e.g., in Cartesian coordinates or Polar coordinates)in the collected image data or in the real world that define at least aportion of the container perimeter 81 (FIG. 4A) of the storage portion85 (FIG. 4A). The set of two-dimensional or three dimensional pointscorrespond to pixel positions in images collected by the first imagingdevice 10, the second imaging device 12, or both. The container module20 may use or retrieve container reference data.

The container reference data comprises one or more of the following:reference dimensions (e.g., length, width, height), volume, referenceshape, drawings, models, layout, and configuration of the container 85,the container perimeter 81, the container edges 181; referencedimensions, reference shape, drawings, models, layout, and configurationof the entire storage portion 93 of receiving vehicle; storage portionwheelbase, storage portion turning radius, storage portion hitchconfiguration of the storage portion 93 of the receiving vehicle; anddistance between hitch pivot point and storage portion wheelbase. Thecontainer reference data may be stored and retrieved from the datastorage device 19 (e.g., non-volatile electronic memory). For example,the container reference data may be stored by, retrievable by, orindexed by a corresponding receiving vehicle identifier in the datastorage device 19 of the transferring vehicle system 11. For eachreceiving vehicle identifier, there can be a corresponding uniquecontainer reference data stored therewith in the data storage device 19.

In one embodiment, the transferring vehicle 91 receives a data messagefrom the receiving vehicle 79 in which a vehicle identifier of thereceiving vehicle is regularly (e.g., periodically transmitted). Inanother embodiment, the transferring vehicle 91 interrogates thereceiving vehicle 79 for its vehicle identifier or establishes acommunications channel between the transferring vehicle 91 and thereceiving vehicle 79 in preparation for unloading via the wirelesscommunication devices 48, 148. In yet another embodiment, the receivingvehicle transmits its vehicle identifier to the transferring vehicle 91when the receiving vehicle 79 approaches the transferring vehicle withina certain radial distance. In still another embodiment, only one knownconfiguration of receiving vehicle 79 is used with a correspondingtransferring vehicle 91 and the container reference data is stored orsaved in the data storage device 19. In the latter embodiment, thetransferring vehicle is programmed, at least temporarily, solely forreceiving vehicles with identical containers, which are identical indimensions, capacity, proportion and shape.

In one configuration, the container module 18 identifies the position ofthe controller as follows. If the linear orientation of a set of pixelsin the collected image data conforms to one or more edges 181 of theperimeter 81 (FIG. 4A) of the container 85 (FIG. 4A) as prescribed bythe container reference data, the position of the container has beenidentified. A target zone, central region or central zone of thecontainer opening 83 of the container 85 can be identified by dividing(by two) the distance (e.g., shortest distance or surface normaldistance) between opposite sides of the container, or by identifyingcorners of the container and where diagonal lines that intercept thecorners intersect, among other possibilities. In one configuration, thecentral zone may be defined as an opening (e.g., circular, elliptical orrectangular) in the container with an opening surface area that isgreater than or equal to the cross-sectional surface area of the spoutend by a factor of at least two, although other surface areas fallwithin the scope of the claims.

The spout module 22 identifies one or more of the following: (1) thespout pixels on at least a portion of the spout 89 (FIG. 4A), or (2)spout end pixels that are associated with the spout end 87 of the spout89 (FIG. 4A). The spout module 22 may use color discrimination,intensity discrimination, or texture discrimination to identifybackground pixels from one or more selected spout pixels with associatedspout pixel patterns or attributes (e.g., color or color patterns (e.g.,Red Green Blue (RGB) pixel values), pixel intensity patterns, texturepatterns, luminosity, brightness, hue, or reflectivity) used on thespout 89 or on the spout end 87 of the spout 89 for identificationpurposes.

The alignment module 24, the master controller 59, or both estimate ordetermine motion commands at regular intervals to maintain alignment ofthe spout 56 over the central zone, central region or target of thecontainer 85 for unloading agricultural material. The alignment module24, the master controller 59, or both, may send commands or requests tothe transferring vehicle 91 with respect to its speed, velocity orheading to maintain alignment of the position of the transferringvehicle 91 with respect to the receiving vehicle 79. For example, thealignment module 24 may transmit a request for a change in a spatialoffset between the vehicles to the master controller 24. In response,the master controller 59 or the coordination module 57 (FIG. 1)transmits a steering command or heading command to the steeringcontroller 32, a braking or deceleration command to a braking system 34,and a propulsion, acceleration or torque command to a propulsioncontroller 40 to achieve the target spatial offset or change in spatialoffset. Further, similar command data may be transmitted via thewireless communication devices 48, 148 to the receiving vehicle 79 forobservational purposes or control of the receiving vehicle via itssteering system controller 32, its braking controller 36, and itspropulsion controller 40 of the system 411 of FIG. 3B.

In another configuration, the alignment module 24 or image processingmodule 18 may regularly or periodically move, adjust or rotate thetarget zone or central zone during loading of the container 85 of thereceiving vehicle 79 to promote even filling, a uniform height, oruniform distribution of the agricultural material in the entirecontainer 85, where the image processing module 18 identifies the fillstate of the agricultural material in the image data from the materialprofile module 27 or receives fill state data from distributed fillstate sensors 149 in FIG. 3A (associated with the container 85) via thewireless communication devices 148, 149.

The imaging module 18 may comprise material profile module 27 or a filllevel sensor for detecting a one-dimensional, two-dimensional orthree-dimensional representation of the fill level or volumetricdistribution of the agricultural material in the container 85 or storageportion 93. For example, FIG. 5C shows various illustrativetwo-dimensional representations of the fill state of the container 85,or the distribution of agricultural material in the container 85, whereFIG. 5C will be described later in detail.

In one configuration illustrated in FIG. 1, the coordination module 57or the steering controller 32 adjusts the relative position (of offset)of the transferring vehicle 91 to the receiving vehicle 79. Thealignment module 24, the coordination module 57 and the auger rotationsystem 16 may control the relative position of the spout 89 or the spoutend 87 to the container perimeter 81 to achieve an even fill to thedesired fill level. For example, rotation actuator 210 or the augerrotation system 16 may adjust the spout angle (e.g., a first spout angle(α), a second spout angle (β), or a compound angle (α and β) that thespout 89 makes with respect to a reference axis or reference coordinatesystem associated with the transferring vehicle 91 or a generallyvertical plane associated with the direction of travel of thetransferring vehicle 91, where the spout 89 meets and rotates withrespect to the transferring vehicle 91.

The spout end 87 may be adjusted for unloading agricultural material byshifting its spout angle or spout position, within the containerperimeter 81 and a tolerance clearance from the container perimeter 81within the container 85. The spout end 87 may be adjusted by varioustechniques that may be applied alternately, or cumulatively. Under afirst technique, the alignment module 24 adjusts the spout end 87 forunloading agricultural material by shifting its spout angle (e.g., afirst spout angle (α), a second spout angle (β), or both.) Under asecond technique, the alignment module 24 requests (or commands) thecoordination module 57 to adjust the fore/aft offset adjustment (Φ orφ), the lateral adjustment (Δ), or both, where the coordination module57 manages or choreographs the relative fore/aft offset and lateraloffset between the transferring vehicle 91 and receiving vehicle 79.Under a third technique, the alignment module 24 primarily adjusts thespout end 87 for unloading agricultural material by shifting its spoutangle and the coordination module 57 secondarily and regularly (e.g.,periodically) moves the fore/aft offset and the lateral offset byfore/aft offset adjustment (Φ or φ), the lateral adjustment (Δ),respectively, to achieve a uniform fill state or level loading of thecontainer with the agricultural material. Accordingly, the spout end 87may be adjusted regularly (e.g., in a matrix of one or more rows orcolumns of preset offset positions) for unloading agricultural materialby shifting the spatial relationship between the transferring vehicle 91and the receiving vehicle 79 by a fore and aft offset or a lateraloffset to achieve a target alignment or desired even distribution offilling the container 85 or storage portion 93 with agriculturalmaterial, while using the spout angle adjustment for fine tuning of thedistribution of the agricultural material within the container (e.g.,from each position within the matrix).

In the image processing module 18, the arbiter 25 comprises an imagedata evaluator. For example, the arbiter 25 may comprise an evaluator, ajudging module, Boolean logic circuitry, an electronic module, asoftware module, or software instructions for determining whether to usethe first image data, the second image data, or both for alignment of arelative position of the spout and the container perimeter (or alignmentof the spatial offset between the vehicles) based on evaluation ofmaterial variation of intensity of pixel data or material variation inambient light conditions during a sampling time interval.

A mode controller 225 is coupled to the data bus (e.g., 60). The modecontroller 225 may comprise a perception quality evaluator, a judgingmodule, Boolean logic circuitry, an electronic module, a softwaremodule, or software instructions for determining whether to operate themachine-vision-augmented guidance system (e.g., 11, 111, or 411) in: (1)an operator-directed manual mode in which one or more human operatorssteer the receiving vehicle 79, the transferring vehicle 91 or bothduring transfer of agricultural material from the transferring vehicle91 to the steering vehicle; (2) an automated mode in which the receivingvehicle 79, the transferring vehicle 91 or both are steered and alignedautomatically during transfer of agricultural material from thetransferring vehicle 91 to the receiving vehicle 79; or (3) a partiallyautomated mode in which one or more operators supervise and can overridethe automated steering and alignment of the transferring vehicle and thereceiving vehicle. For example, the mode controller 225 may determinewhether to use an automated control mode of the spout or anoperator-directed manual control mode of the spout based on a firstoperational status of a first location determining receiver 42associated with the transferring vehicle 91, a second operational statusof a second location determining receiver 142 associated with thereceiving vehicle 79, and a third operational status of the firstimaging device.

In one embodiment, the mode controller 225 comprises a perceptionquality evaluator that evaluates the functionality, diagnostics,performance, tests or quality of one or more location determiningreceivers 42, 142, imaging devices 10, 12, range finders, odometrysensors 440, dead-reckoning sensors, inertial sensors 442, navigationsensors, or other perception sensors. In one illustrative example, thefirst operational status is acceptable if the first location determiningreceiver 42 provides reliable position data that meets or exceeds adilution of precision threshold or another navigation satellitereliability measure during a sampling period; the second operationalstatus is acceptable if the second location determining receiver 142provides reliable position data that meets or exceeds a dilution ofprecision threshold or another navigation satellite reliability measure(e.g., total equivalent user range error) during a sampling period; andthe third operational status is acceptable if the first imaging device10 provides reliable image data in which the container module 20 orspout module 22 (e.g., or the respective edge detection modules therein)are capable of any of the following: (1) reliably identifying orresolving one or more edges of spout 89 or container perimeter 81 in thecollected image data during a sampling time period, or (2) reliablyidentifying on a time percentage basis (e.g., at least 99.99% of thetime) one or more reference objects (e.g., a reference pattern orreference image on the spout or receiving vehicle 79) or objects in theimage data.

Dilution of precision provides a figure of merit of the performance of alocation determining receiver 42, 142 that uses a satellite navigationsystem, such as the Global Positioning System (GPS) or Global NavigationSatellite System (GLONASS). Dilution of precision captures thetime-varying impact of spatial geometry and separation between alocation determining receiver 42, 142 and satellites signals that arereceived by the location determining receiver, as opposed to clockerrors, ionospheric errors, multipath errors, and other errors. Theprecision in pseudo-range estimate to each satellite can affect theaccuracy of the determination of a three dimensional position estimateand time estimate of the location determining receiver 42, 142. Ifreceivable navigation satellites are spatially too close together inorbit for a given location determining receiver a particular time,accuracy of the position estimate may be compromised and the dilution ofprecision value can be higher than normal or acceptable.

A master controller 59 is coupled to the data bus 58, 60. In oneembodiment, the master controller 59 comprises an auto-guidance module55 and coordination module 57. The auto-guidance module 55 or mastercontroller 59 can control the transferring vehicle 91 in accordance withlocation data from the first location determining receiver 42 and a pathplan or desired vehicle path (e.g., stored in data storage 19). Theauto-guidance module 55 or master controller 59 sends command data tothe steering controller 32, the braking controller 36 and the propulsioncontroller 40 to control the path of the transferring vehicle to trackautomatically a path plan or to track manually steered course of anoperator via the user interface 44 or steering system 30.

The coordination module 57 may facilitate alignment of movement (e.g.,choreography) between the transferring vehicle 91 (FIG. 4A) and thereceiving vehicle 79 (FIG. 4A) during unloading or transferring ofagricultural material between the vehicles. For example, thecoordination module 57 may facilitate maintenance of a uniform lateraloffset (Δ in FIG. 4) and a uniform fore/aft offset (Φ or φ in FIG. 4)between the vehicles during unloading of the agricultural material,subject to any adjustments for attainment of a uniform distribution ofmaterial in the container 85. Collectively, the uniform lateral offsetand uniform for/aft offset may be referred to as a uniform spatialoffset. In certain embodiments, maintenance of the lateral offset andfore/aft offset, or coordination of any shift in the lateral offset andfore/aft offset (e.g., pursuant to a two-dimensional matrix ofpre-established positions (x, y points) for uniform loading of arespective particular container or storage portion), is a necessary ordesired precondition to implementing spout angle adjustment of the spout89 or spout end 87 by the alignment module 24.

In one embodiment in a leader mode, the transferring vehicle 91 issteered by the auto-guidance module 55 or the steering controller 32 inaccordance with path plan, or by a human operator. The master controller59 or coordination module 57 controls the receiving vehicle 79 in afollower mode via the slave controller 159, where the transferringvehicle 91 operates in the leader mode. If the transferring vehicle 91operates in an automated mode or auto-steering mode, the mastercontroller 59 provides command data locally to the steering controller32, braking controller 36, and propulsion engine controller 40 of thetransferring vehicle 91. Such command data can be normalized (orscaled), time stamped, and communicated to the receiving vehicle 79 viawireless communication devices 48, 148 for processing by the slavecontroller 159. Alternatively, the velocity, acceleration, and headingdata of the transferring vehicle 91 is communicated to the receivingvehicle 79 via the wireless communications devices 48, 148 to enable toreceiving vehicle to follow the path of the transferring vehicle 91(e.g., with a minimal time delay). In an automated mode and in aleader-follower mode, the receiving vehicle 79, the transferring vehicleor both are steered and aligned automatically during transfer ofagricultural material from the transferring vehicle 91 to the receivingvehicle 79.

The image processing module 18 provides image data to a user interfaceprocessing module 26 that provides, directly or indirectly, statusmessage data and performance message data to a user interface 44. Asillustrated in FIG. 1, the image processing module 18 communicates witha vehicle data bus 31 (e.g., Controller Area Network (CAN) data bus).

In one embodiment, a location determining receiver 42, a first wirelesscommunications device 48, a vehicle controller 46, a steering controller32, a braking controller 36, and a propulsion controller 40 are capableof communicating over the vehicle data bus 31. In turn, the steeringcontroller 32 is coupled to a steering system 30 of the transferringvehicle 91; the braking controller 36 is coupled to the braking system34 of the transferring vehicle 91; and the propulsion controller 40 iscoupled to the propulsion system 38 of the transferring vehicle 91.

In FIG. 1, the steering system 30 may comprise an electrically-drivensteering system, an electro-hydraulic steering system, a gear drivensteering system, a rack and pinion gear steering system, or anothersteering system that changes the heading of the vehicle or one or morewheels of the vehicle. The braking system 34 may comprise a regenerativebraking system, an electro-hydraulic braking system, a mechanicalbreaking system, or another braking system capable of stopping thevehicle by hydraulic, mechanical, friction or electrical forces. Thepropulsion system 38 may comprise one or more of the following: (1) thecombination of an electric motor and an electric controller, (2)internal combustion engine that is controlled by an electronic fuelinjection system or another fuel metering device that can be controlledby electrical signals, or (3) a hybrid vehicle in which an internalcombustion engine drives a electrical generator, which is coupled to oneor more electric drive motors.

The system 11 facilitates the transfer of agricultural material from thetransferring vehicle 91 (e.g., a harvesting vehicle) to a receivingvehicle 79. The system 11 comprises a receiving vehicle 79 with apropelled portion for propelling the receiving vehicle 79 and a storageportion 93 for storing agricultural material. A stereo imaging device,such as the first imaging device 10, faces towards the storage portion93 of the receiving vehicle 79. As shown in FIG. 1, the first imagingdevice 10 and the optional second imaging device 12 are mounted on thetransferring vehicle 79, consistent with FIG. 4.

One or more imaging devices 10, 12 are arranged to collect image data. Acontainer module 20 identifies a container perimeter 81 of the storageportion 93 in the collected image data. The storage portion 93 has anopening inward from the container perimeter for receipt of theagricultural material. A spout module 22 is configured to identify aspout 89 (FIG. 4A) of the transferring vehicle 91 in the collected imagedata. An alignment module 24 is adapted for determining the relativeposition of the spout 89 and the container perimeter 81 (FIG. 4A) andfor generating command data to the transferring vehicle or the propelledportion 75 of the receiving vehicle 79 to steer the storage portion 93in cooperative alignment such that the spout 89 is aligned within acentral zone 83 of the container perimeter 81. A steering controller 32is associated with a steering system 30 of the propelled portion forsteering the receiving vehicle 79 in accordance with the cooperativealignment.

In one embodiment, an optional mast controller 674, indicated by dashedlines, is coupled to the vehicle data bus 60, the implement data bus 58in FIGS. 1, 2 and FIG. 3A), or the image processing module 18 to controlan optional adjustable mast 573 for mounting and adjustably positioningthe first imaging device 10, the second imaging device 12, or both. Themast controller 674 is adapted to change the orientation or height aboveground of the first imaging device 10, the second imaging device 12 orboth, where the orientation may be expressed as any of the following: atilt angle, a pan angle, a down-tilt angle, a depression angle, or arotation angle.

In one illustrative embodiment of a machine-vision guidance system(e.g., 11, 111, 311) that has an adjustable mast 573, at least oneimaging device 10, 12 faces towards the storage portion 93 of thereceiving vehicle 79 and collects image data. For example, via data fromthe mast controller 674 the adjustable mast 573 is capable of adjustinga height of the imaging device 10, 12 within a height range, adjusting adown-tilt angle of the imaging device 10, 12 within a down-tilt angularrange, and a rotational angle or pan angle within a pan angular range.The image processing module 18 is adapted or programmed (e.g., withsoftware instructions or code) to determine whether to adjust the heightof the imaging device 10, 12 or whether to decrement or increment thedown-tilt angle of the imaging device 10, 12 based on evaluation ofmaterial variation of intensity of pixel data or material variation inambient light conditions (e.g., from the optical sensor 110, 112) duringa sampling time interval. Under certain operating conditions, such asoutdoor ambient light conditions, increasing or incrementing thedown-tilt angle may increase the quality level of the collected imagedata or reduce variation in the intensity of the image data to below athreshold variation level. Reduced variation in intensity of the imagedata or reduced collection of dust or debris on a lens of the imagingdevice are some advantages that can be realized by increasing oradjusting down-tilt angle of the imaging device 10, 12, for example. Aspreviously noted, a container module 19 can identify a containerperimeter 81 of the storage portion 93 in the collected image data.Similarly, a spout module 22 can identify a spout of the transferringvehicle 91 in the collected image data. An alignment module 24determines the relative position of the spout and the containerperimeter 81 and generates command data to the propelled portion 75 tosteer the storage portion 93 in cooperative alignment such that thespout 89, or spout end 87, is aligned within a target zone or centralzone of the container perimeter 81. A steering controller 32 isassociated with a steering system 30 of the propelled portion 75 forsteering the receiving vehicle 79 in accordance with the cooperativealignment.

In one illustrative embodiment of a machine-vision guidance system withthe adjustable mast 573, the image processing module 18 sends a datamessage to a mast controller 674 (or the adjustable mast 573) toincrement or increase the down-tilt angle if the material variation ofintensity of pixel data or if the material variation in ambient lightconditions exceeds a threshold variation level during a sampling timeinterval. For example, the image processing module 18 sends a datamessage to a mast controller 674 to increment or increase the down-tiltangle at discrete levels (e.g., one degree increments or decrements)within an angular range of approximately negative ten degrees toapproximately negative twenty-five degrees from a generally horizontalplane.

In one configuration, a user interface 44 is arranged for enteringcontainer reference data or dimensional parameters related to thereceiving vehicle 79 into vehicle module 1000. For example, thecontainer reference data or dimensional parameters comprise a distancebetween a trailer hitch or pivot point (which interconnects thepropulsion unit 75 and the storage portion 93) and front wheelrotational axis of the storage portion 93 of the receiving vehicle 79.

In an alternate embodiment, in FIG. 1 and FIG. 4A the first imagingdevice 10 comprises a monocular imaging device and the second imagingdevice 12 comprises a monocular imaging device that provides firstmonocular image data and second monocular image data, respectively. Theimage processing module 18 or system (11, 111) can create a stereo imagefrom the first monocular image data (e.g., right image data) and thesecond monocular image data (e.g., left image data) with reference tothe relative position and orientation of the first imaging device 10 andthe second imaging device 12. The image processing module 18 determines:(1) at least two points on a common visual axis that bisects the lensesof both the first imaging device 10 and the second imaging device 12,and (2) a linear spatial separation between the first imaging device 10and the second imaging device 12, where the first field of view 77 ofthe first imaging device 10 and the second field of view 177 of thesecond imaging device 12 overlap, at least partially, to capture thespout 89, the spout end 87 and the container perimeter 81 in thecollected image data.

In an alternate embodiment, FIGS. 1 and 2 further comprises an optionalodometer sensor 440, and an optional inertial sensor 442, as illustratedby the dashed lines. The odometer sensor 440 may comprise a magneticrotation sensor, a gear driven sensor, or a contactless sensor formeasuring the rotation of one or more wheels of the transferring vehicle79 to estimate a distance traveled by the transferring vehicle during ameasurement time period, or a ground speed of the transferring vehicle79. The odometry sensor 440 may be coupled to the vehicle data bus 60 oran implement data bus 58. The inertial sensor 442 may comprise one ormore accelerometers, gyroscopes or other inertial devices coupled to thevehicle data bus 60 or an implement data bus 58. The optional odometrysensor 440 and the optional inertial sensor 442 may augment orsupplement position data or motion data provided by the first locationdetermining receiver 42.

The system 11 of FIG. 1 is well suited for use on a combine or harvesteras the transferring vehicle 91. The system 11 of FIG. 1 may communicateand cooperate with a second system 411 on the receiving vehicle 79(e.g., as illustrated in FIG. 3B) to coordinate the relative alignmentof the transferring vehicle 91 and the receiving vehicle 79 duringunloading or transferring of material from the transferring vehicle 79.Like reference numbers in FIG. 1 and FIG. 2 indicate like elements.

The vision-augmented guidance system 111 of FIG. 2 is similar to thesystem 11 of FIG. 1; except that the system 111 of FIG. 2 furthercomprises an implement data bus 58, a gateway 29, and vehiclecontrollers 50, 54 coupled to the vehicle data bus 60 for the lights 14and spout 89. The vehicle controller 50 controls the lights 14; thespout controller 54 controls the spout 89 via a servo-motor, electricmotor, or an electro-hydraulic mechanism for moving or adjusting theorientation or spout angle of the spout 89, or its spout end 87. In oneconfiguration, the implement data bus 58 may comprise a Controller AreaNetwork (CAN) implement data bus. Similarly, the vehicle data bus 60 maycomprise a controller area network (CAN) data bus. In an alternateembodiment, the implement data bus 58, the vehicle data bus 60, or bothmay comprise an ISO (International Organization for Standardization)data bus or ISOBUS, Ethernet or another data protocol or communicationsstandard.

The gateway 29 supports secure or controlled communications between theimplement data bus 58 and the vehicle data bus 60. The gateway 29comprises a firewall (e.g., hardware or software), a communicationsrouter, or another security device that may restrict or prevent anetwork element or device on the implement data bus 58 fromcommunicating (e.g., unauthorized communication) with the vehicle databus 60 or a network element or device on the vehicle data bus 31, unlessthe network element or device on the implement data bus 58 follows acertain security protocol, handshake, password and key, or anothersecurity measure. Further, in one embodiment, the gateway 29 may encryptcommunications to the vehicle data bus 60 and decrypt communicationsfrom the vehicle data bus 60 if a proper encryption key is entered, orif other security measures are satisfied. The gateway 29 may allownetwork devices on the implement data bus 58 that communicate via anopen standard or third party hardware and software suppliers, whereasthe network devices on the vehicle data bus 60 are solely provided bythe manufacturer of the transferring vehicle (e.g., self-propelledforage harvester) or those authorized by the manufacturer.

In FIG. 2, a first location determining receiver 42, a user interface44, a user interface processing module 26, and the gateway 29 arecoupled to the implement data bus 58, although in other embodiments suchelements or network devices may be connected to the vehicle data bus 60.controllers 50, 54 are coupled to the vehicle data bus 60. In turn, thecontrollers 50, 54 are coupled, directly or indirectly, to lights 14 onthe transferring vehicle 91 and the spout 89 of the transferring vehicle91 (e.g., self-propelled forage harvester). Although the system of FIG.2 is well suited for use or installation on a self-propelled forageharvester, the system of FIG. 2 may also be applied to combines,harvesters or other heavy equipment.

The system 11 of FIG. 1 and the system 111 of FIG. 2 apply to thetransferring vehicle 91, whereas the system of FIGS. 3A and 3B appliesto the receiving vehicle 79. Like reference numbers in FIGS. 1, 2, 3A,and 3B indicate like elements. As previously noted, the transferringvehicle 91 may comprise a combine, harvester, self-propelled harvester,vehicle or heavy equipment that collects or harvests material fortransfer to the receiving vehicle 79. In one embodiment, the receivingvehicle 79 may comprise a propelled portion 75 (FIGS. 4A and 5A) and astorage portion 93 (FIGS. 4A and 5A) for storing the materialtransferred from the transferring vehicle 91. The receiving vehicle 79may comprise the combination of a tractor and a grain cart or wagon,where the tractor is an illustrative example of the propelled portion 75and where the grain cart is an illustrative example of the storageportion 93.

In one embodiment of FIG. 5D, both the transferring vehicle 91 and thereceiving vehicle 79 may be moving forward at approximately the samevelocity and heading (e.g., within a tolerance or error of the controlsystems during harvesting), where the relative position of the receivingvehicle 79 is generally fixed or constant with respect to each position(502, 504) in the matrix 500 that the transferring vehicle 91 canoccupy.

In an alternate embodiment, the receiving vehicle 79 may be shown asoccupying a two dimensional matrix (e.g., 3×3 matrix, with three columnsand three rows) of possible offset positions, while the position of thetransferring vehicle 91 is generally fixed or constant with respect toeach position of matrix that the receiving vehicle 79 could occupy. Asdirected by any of the systems (11, 111, 311) in the alternateembodiment, the imaging processing module 18, or the master controller159 of the receiving vehicle 79 can shift to any unoccupied or otherpossible offset positions within the matrix to promote or facilitate aneven distribution of agricultural material within the container 85 orstorage portion of the receiving vehicle 79.

In FIG. 6 and FIG. 7, each of the blocks or modules may representsoftware modules, electronic modules, or both. Software modules maycontain software instructions, subroutines, object-oriented code, orother software content. The arrows that interconnect the blocks ormodules of FIG. 6 show the flow of data or information between theblocks. The arrows may represent physical communication paths or virtualcommunication paths, or both. Physical communication paths meantransmission lines or one or more data buses for transmitting, receivingor communicating data. Virtual communication paths mean communication ofdata, software or data messages between modules.

FIG. 6 is a block diagram that shows the imaging processing module 18and the container module 20 in greater detail than FIG. 1. Likereference numbers in FIG. 1, FIG. 6, and FIG. 7 indicate like elements.As illustrated in FIG. 6, the first imaging device 10, the secondimaging device 12, or both, provide input of raw stereo camera images(or raw image data) to the image rectification module 101. In turn, theimage rectification module 101 communicates with the disparity imagegenerator 103 and the edge detector 105. The edge detector 105 providesan output to the linear Hough transformer 107. The outputs of thedisparity image generator 103 and the linear Hough transformer 107 areprovided to the container localizer 111. The container localizer 111 mayaccess or receive stored (a priori) hitch and container measurements,container dimensions, container volume or other receiving vehicle datafrom the data manager 109. In one embodiment, the container localizer111 may receive or access and an estimate of the tongue angle (betweenthe propulsion portion 75 and the storage portion 93 of the receivingvehicle 79) from the angle estimator 113 (e.g., Kalman filter) andstored hitch and container measurements.

In the another embodiment, the image rectification module 101 providesimage processing to the collected image data or raw stereo images toreduce or remove radial lens distortion and image alignment required forstereo correspondence. The radial lens distortion is associated with theradial lenses of the first imaging device 10, the second imaging device12, or both. The input of the image rectification module 101 is rawstereo image data, whereas the output of the image rectification module101 is rectified stereo image data.

In one illustrative embodiment, the image rectification module 101eliminates or reduces any vertical offset or differential between a pairof stereo images of the same scene of the image data. Further, the imagerectification module 101 can align the horizontal component (orhorizontal lines of pixels of the stereo images) to be parallel to thescan lines or common reference axis of each imaging device (e.g., leftand right imaging device) within the first and second imaging devices10, 12. For example, the image rectification module may use histogramequalization and calibration information for the image processingdevices 10, 12 to achieve rectified right and left images of the stereoimage. The rectified image supports efficient processing and readyidentification of corresponding pixels or objects within the image inthe left image and right image of a common scene for subsequent imageprocessing (e.g., by the disparity image generator 103).

In one configuration, the disparity image generator 103 applies a stereomatching algorithm or disparity calculator to collected stereo imagedata, such as the rectified stereo image data outputted by the imagerectification module 101. The stereo matching algorithm or disparitycalculator may comprise a sum of absolute differences algorithm, a sumof squared differences algorithm, a consensus algorithm, or anotheralgorithm to determine the difference or disparity for each set ofcorresponding pixels in the right and left image (e.g., along ahorizontal axis of the images or parallel thereto).

In an illustrative sum of the absolute differences procedure, the rightand left images (or blocks of image data or rows in image data) can beshifted to align corresponding pixels in the right and left image. Thestereo matching algorithm or disparity calculator determines a disparityvalue between corresponding pixels in the left and right images of theimage data. For instance, to estimate the disparity value, each firstpixel intensity value of a first subject pixel and a first sum of thefirst surrounding pixel intensity values (e.g., in a block or matrix ofpixels) around the first pixel is compared to each corresponding secondpixel intensity value of second subject pixel and a second sum of thesecond surrounding pixel intensity values (e.g., in a block or matrix ofpixels) around the second pixel. The disparity values can be used toform a disparity map or image for the corresponding right and left imagedata.

The image processing module 18, or container localizer 111, estimate adistance or range from the first imaging device 10, the second imagingdevice 12, or both to the pixels or points lying on the containerperimeter 81, on the container edge 181, on the spout 89, on the spoutend 87, or on any other linear edge, curve, ellipse, circle or objectidentified by the edge detector 105, the linear Hough transformer 107,or both. For example, the image processing module 18 may use thedisparity map or image to estimate a distance or range from the firstimaging device 10, the second imaging device 12, or both to the pixelsor points lying on the container perimeter 81, the container edges 181,the container opening 83, in the vicinity of any of the foregoing items,or elsewhere.

In one embodiment, the container module 20 comprises: (1) an edgedetector 105 for measuring the strength or reliability of one or moreedges 181, or points on the container perimeter 81 in the image data;(2) a linear Hough transformer 107 for identifying an angle and offsetof candidate linear segments in the image data with respect to areference point on an optical axis, reference axis of the one or moreimaging devices 10, 12; (3) a container localizer 111 adapted to usespatial and angular constraints to eliminate candidate linear segmentsthat cannot logically or possibly form part of the identified linearsegments of the container perimeter 81, or points on the containerperimeter 81; and (4) the container localizer 111 transforms thenon-eliminated, identified linear segments, or identified points, intotwo or three dimensional coordinates relative to a reference point orreference frame of the receiving vehicle and harvesting vehicle.

The edge detector 105 may apply an edge detection algorithm to rectifiedimage data from the image rectification module 101. Any number ofsuitable edge detection algorithms can be used by the edge detector 105.Edge detection refers to the process of identifying and locatingdiscontinuities between pixels in an image or collected image data. Forexample, the discontinuities may represent material changes in pixelintensity or pixel color which defines boundaries of objects in animage. A gradient technique of edge detection may be implemented byfiltering image data to return different pixel values in first regionsof greater discontinuities or gradients than in second regions withlesser discontinuities or gradients. For example, the gradient techniquedetects the edges of an object by estimating the maximum and minimum ofthe first derivative of the pixel intensity of the image data. TheLaplacian technique detects the edges of an object in an image bysearching for zero crossings in the second derivative of the pixelintensity image. Further examples of suitable edge detection algorithmsinclude, but are not limited to, Roberts, Sobel, and Canny, as are knownto those of ordinary skill in the art. The edge detector 105 may providea numerical output, signal output, or symbol, indicative of the strengthor reliability of the edges 181 in field. For example, the edge detectormay provide a numerical value or edge strength indicator within a rangeor scale or relative strength or reliability to the linear Houghtransformer 107.

The linear Hough transformer 107 receives edge data (e.g., an edgestrength indicator) related to the receiving vehicle and identifies theestimated angle and offset of the strong line segments, curved segmentsor generally linear edges (e.g., of the container 85, the spout 89, thespout end 87 and opening 83) in the image data. The estimated angle isassociated with the angle or compound angle (e.g., multidimensionalangle) from a linear axis that intercepts the lenses of the firstimaging device 10, the second image device 12, or both. The linear Houghtransformer 107 comprises a feature extractor for identifying linesegments of objects with certain shapes from the image data. Forexample, the linear Hough transformer 107 identifies line equationparameters or ellipse equation parameters of objects in the image datafrom the edge data outputted by the edge detector 105, or Houghtransformer 107 classifies the edge data as a line segment, an ellipse,or a circle. Thus, it is possible to detect containers or spouts withgenerally linear, rectangular, elliptical or circular features.

In one embodiment, the data manager 109 supports entry or selection ofcontainer reference data by the user interface 44. The data manager 109supports entry, retrieval, and storage of container reference data, suchas measurements of cart dimensions, by the image processing module 18 togive spatial constraints to the container localizer 111 on the linesegments or data points that are potential edges 181 of the cart opening83.

In one embodiment, the angle estimator 113 may comprise a Kalman filteror an extended Kalman filter. The angle estimator 113 estimates theangle of the storage portion 93 (e.g., cart) of the receiving vehicle 79to the axis of the direction of travel of the propelled portion 75(e.g., tractor) of the receiving vehicle 79. The angle estimator 113(e.g., Kalman filter) provides angular constraints to the containerlocalizer 111 on the lines, or data points, that are potential edges 181of the container opening 83. In configuration, the angle estimator 113or Kalman filter is coupled to the localizer 111 (e.g., containerlocalizer). The angle estimator filter 113 outputs, or is capable ofproviding, the received estimated angle of the storage portion 93relative to the axis of the direction of travel of the propellingportion 75 of the vehicle.

The localizer 111 is adapted to receive measurements of dimensions ofthe container perimeter 81 or the storage portion 93 of the vehicle tofacilitate identification of candidate linear segments that qualify asidentified linear segments of the container perimeter 81. In oneembodiment, the localizer 111 is adapted to receive an estimated angleof the storage portion 93 relative to the propelling portion 75 of thevehicle to facilitate identification of candidate linear segments thatqualify as identified linear segments of the container perimeter 81. Thelocalizer 111 uses spatial and angular constraints to eliminatecandidate lines in the image data that cannot be possibly or logicallypart of the container opening 83 or container edges 181, then selectspreferential lines (or data points on the container edge 81) as the mostlikely candidates for valid container opening 83 (material therein) orcontainer edges 181. The localizer 111 characterizes the preferentiallines as, or transformed them into, three dimensional coordinatesrelative to the vehicle or another frame of reference to represent acontainer perimeter of the container 85.

FIG. 7 is a block diagram that shows the image processing module 18 andthe spout module 22 in greater detail than FIG. 1. Like referencenumbers in FIGS. 1, 2, 3A, 3B, 6 and 7 indicate like elements. In FIG.7, the image rectification module 101 communicates with the disparityimage generator 103 and the spout classifier 121. In turn, the spoutclassifier 121 provides an output to the spout localizer 125. The spoutlocalizer 125 accesses or receives the spout position from angle sensor115 or the spout position estimator 123 (or spout angle (α) with respectto the transferring vehicle direction of travel or vehicle referenceframe), stereo correspondence data from the disparity image generator103, and the output data from the spout classifier 121.

In one embodiment, the spout (identification) module 22 comprises aspout classifier 121 that is configured to identify candidate pixels inthe image data based at least one of reflectivity, intensity, color ortexture features of the image data (or pixels), of the rectified imagedata or raw image data, where the candidate pixels represent a portionof the spout 89 or spout end 87. The spout localizer 125 is adapted toestimate a relative position of the spout 89 to the imaging device basedon the classified, identified candidate pixels of a portion of the spout89. The spout localizer 125 receives an estimated combine spout positionor spout angle (α) relative to the mounting location of the imagingdevice, or optical axis, or reference axis of one or more imagingdevices, based on previous measurements to provide constraint data onwhere the spout 89 can be located possibly.

The spout classifier 121 applies or includes software instructions on analgorithm that identifies candidate pixels that are likely part of thespout 89 or spout end 87 based on expected color and texture featureswithin the processed or raw image data. For example, in oneconfiguration the spout end 87 may be painted, coated, labeled or markedwith a coating or pattern of greater optical or infra-red reflectivity,intensity, or luminance than a remaining portion of the spout 89 or thetransferring vehicle. The greater luminance, intensity or reflectivityof the spout end 87 (or associated spout pixels of the image data versusbackground pixels) may be attained by painting or coating the spout end87 with white, yellow, chrome or a lighter hue or shade with respect tothe remainder of the spout 89 or portions of the transferring vehicle(within the field of view of the imaging devices 10, 12.

In one embodiment, the spout position estimator 123 comprises a Kalmanfilter or an extended Kalman filter that receives input of previousmeasurements and container reference data and outputs an estimate of thespout position, spout angle, or its associated error. The spout positionestimator 123 provides an estimate of the combine spout position, orspout angle, or its error, relative to one or more of the following: (1)the mounting location or pivot point of the spout on the transferringvehicle, or (2) the optical axis or other reference axis or point of thefirst imaging device 10, the second imaging device 12, or both, or (3)the axis associated with the forward direction of travel or the headingof the transferring vehicle. The Kalman filter outputs constraints onwhere the spout 89 or spout end 87 can be located, an estimated spoutposition, or a spout location zone or estimated spout position zone. Inone embodiment, the spout position estimator 123 or Kalman filter iscoupled to the spout localizer 125.

The spout localizer 125 takes pixels that are classified as belonging tothe combine auger spout 89 and uses a disparity image (from stereocorrespondence data) to estimate the relative location of the spout tothe first imaging device 10, the second imaging device 12, or both, orreference axis or coordinate system associated with the vehicle.

The flow chart of FIG. 8 begins in block S800. In block S800, theprocess starts. In block S802, the mode controller 225 or system (11,111, 311) decides if the second location determining receiver 142 of thereceiving vehicle provides reliable position data during an evaluationtime period. In block S802, the determination of reliable position dataor acceptable position data may be carried out by the followingtechniques that may be applied separately or cumulatively. Under a firsttechnique of executing step S802, the second location determiningreceiver 142 provides reliable position data or acceptable position dataif the dilution of precision or total equivalent user range error isless than a threshold level during a sampling period.

Under a second technique of executing step S802, the locationdetermining receiver 142 is assumed to provide reliable position data oracceptable position data, unless the receiver provides an error code, adiagnostic code, a fault code, or an alarm.

Under a third technique, the second location determining receiver 142provides reliable position data or acceptable position data, unless itfails to output location data on a certain monitored data port, or failsto provide accurate or reliable location data on a certain monitoreddata port consistent with alignment with a known position landmark orreference position in a work area or field. Under the third technique,the transferring vehicle, receiving vehicle or both are aligned with theknown position landmark or reference position on a regular or periodicbasis (e.g., at start of harvesting or at the beginning of anagricultural task) to check for reliable performance of the locationdetermining receivers (42, 142).

If the second location determining receiver 142 provides reliableposition data during the evaluation time period, the method continueswith block S806. However, if the second location determining receiver142 does not provide reliable position data during the evaluation timeperiod, then the method continues with block S804.

In block S804, the image processing module 18, the arbiter 25, the modecontroller 225, or the system (11, 111, 311) decides if at least oneimaging device (e.g., 10, 12, stereo vision imaging device or a pair ofmonocular imaging devices) is providing reliable image data. In oneexample, the imaging device provides reliable or acceptable image datain which the container module 20 or spout module 22 (e.g., or therespective edge detection modules therein) are capable of one or more ofthe following: (1) reliably identifying or resolving one or more edgesof spout 89 or container perimeter 81 in the collected image data duringa sampling time period, or (2) reliably identifying on a time percentagebasis (e.g., at least 99.99% of the time) one or more reference objects(e.g., a reference pattern or reference image on the spout or receivingvehicle) or objects in the image data. If the imaging device (10, 12) isproviding reliable image data, then the method continues with blockS812. However, if the imaging device (10, 12) is not providing reliableimage data, then the method continues with block S808.

In block S806, the mode controller 225 or the system (11, 111, 311)decides if the first location determining receiver 42 of thetransferring vehicle is providing reliable position data. In block S806,the determination of reliable position data or acceptable position datamay be carried out by the following techniques that may be appliedseparately or cumulatively. Under a first technique of executing stepS802, the first location determining receiver 42 provides reliableposition data or acceptable position data if the dilution of precisionor total equivalent user range error is less than a threshold levelduring a sampling period.

Under a second technique of executing step S802, the first locationdetermining receiver 42 is assumed to provide reliable position data oracceptable position data, unless the receiver provides an error code, adiagnostic code, a fault code, or an alarm.

Under a third technique, the first location determining receiver 42provides reliable position data or acceptable position data, unless itfails to output location data on a certain monitored data port, or failsto provide accurate or reliable location data on a certain monitoreddata port consistent with alignment with a known position landmark orreference position in a work area or field. Under the third technique,the transferring vehicle, receiving vehicle or both are aligned with theknown position landmark or reference position on a regular or periodicbasis (e.g., at start of harvesting or at the beginning of anagricultural task) to check for reliable performance of the locationdetermining receivers (42, 142).

If the first location determining receiver 42 of the transferringvehicle is providing reliable position data, the method continues withstep S810. However, if the first location-determining receiver 42 is notproviding reliable position data, the method continues with or returnsto block S804.

In block S808, the image processing module 18, the arbiter 25, or themode controller 225 determines that the system (11, 111, 311) does notprovide reliable tracking data for automated guidance and alignment ofthe spout 89 and the container 85 of the receiving portion 93.Accordingly, the image processing system 18, arbiter 25, or modecontroller 225 determines that the system (11, 111, 311) shall operatein an operator-directed manual mode for a control time period followingthe evaluation time period.

In block S810, the image processing module 18, the arbiter 25, or themode controller 225 decides if at least one imaging device (e.g., 10,12, stereo vision imaging device or a pair of monocular imaging devices)is providing reliable image data. In one example, the imaging device(10, 12) provides reliable or acceptable image data in which thecontainer module 20 or spout module 22 (e.g., or the respective edgedetection modules therein) are capable of any of the following: (1)reliably identifying or resolving one or more edges of spout 89 orcontainer perimeter 81 in the collected image data during a samplingtime period, or (2) reliably identifying on a time percentage basis(e.g., at least 99.99% of the time) one or more reference objects (e.g.,a reference pattern or reference image on the spout 89 or receivingvehicle 91) or objects in the image data.

If the imaging device (10,12) is providing reliable image data, then themethod continues with block S814. However, if the imaging device is notproviding reliable image data, then the method continues with blockS816.

In block S812, the image processing module 18, mode controller 225, orsystem (11, 111, 311) operates in the manual mode or a partiallyautomated mode. In the partially automated mode, an operator may bepresent in the vehicles for overriding, supplementing or correcting theguidance or alignment data providing by the imaging device (10,12)tracking, the odometry data (via odometry sensor 440), inertial data(via inertial sensor 442) and the angle estimator 113. For example, instep S812 the imaging processing module 18 or mode controller 225 usescamera tracking, odometry data, and the angle estimator 113 (e.g.,Kalman filter). The angle estimator 113 can be used to estimate an anglebetween a propulsion unit and a container of the receiving vehicle.

In block S814, the image processing module 18, mode controller 225, orsystem (11, 111, 311) operates in the automated mode. For example, theimaging processing module 18 or mode controller 225 uses position dataof the location determining receivers (42, 142), tracking data of atleast one imaging device (10, 12), and the angle estimator 113 (e.g.,Kalman filter). The angle estimator 113 can be used to estimate an anglebetween a propulsion portion 75 and a storage portion 93 of thereceiving vehicle.

In block S816, the imaging processing module 18, mode controller 225, orsystem operates in a manual mode or partially automated mode. In thepartially automated mode, one or more operators may be present in thevehicles (79, 91) for overriding, supplementing or correcting theguidance or alignment data providing by the location determiningreceivers (42, 142). For example, the imaging processing module 18 usesposition data of the location determining receivers (42, 142) to guidethe vehicles or to align the spout and the container.

After block S808, S812, S814 or S816, the method of FIG. 8 may continuewith block S817. In block S817, the image processing module 18, modecontroller 225, or system may wait a time period (e.g., a control timeperiod or an evaluation time period) prior to looping or returning tostep S802.

FIG. 9 is a flow chart of a method for facilitating the unloading ofagricultural material from a vehicle or between a transferring vehicle91 and a receiving vehicle 79. The method of FIG. 9 begins in step S900and may use one or more of the following embodiments of the systems 11,111, or 311 previously disclosed herein.

In step S900, the transferring vehicle 91 (e.g., harvester or combine)stores agricultural material in a storage portion (e.g., grain bin) ofthe transferring vehicle 91. For example, the transferring vehicle maystore the agricultural material in the storage portion of thetransferring vehicle 91 as the transferring vehicle 91 moves forward andharvests crop in a field. As the storage portion or storage vessel(e.g., grain tank) of the transferring vehicle 91 becomes full or nearcapacity, the receiving vehicle may move along one side of the movingtransferring vehicle 91 for unloading of the agricultural material(e.g., consistent with or similar to the illustration of FIGS. 4A and5A).

In step S902, the first imaging device 10 faces toward the storageportion of the receiving vehicle 79 (e.g., grain cart) and collectsfirst image data (e.g., first stereo image data, first monocular imagedata, or a right image of a stereo image). For example, the firstimaging device 10 may be mounted on the transferring vehicle 91 facingthe receiving vehicle 79 facing the container 85. In one embodiment, thefirst imaging device 10 has first field of view (77 in FIG. 4A) of thestorage portion of the receiving vehicle 79.

In an alternative embodiment, the first imaging device 10 comprises amonocular imaging device that provides a first image section (e.g., leftimage) of stereo image data of a scene or an object.

In step S904, where present, the optional second imaging device 12 facestoward the storage portion 93 of the receiving vehicle 79 (e.g., graincart) and collects second image data (e.g., second stereo image data,second monocular image data, or a left image of a stereo image). Forexample, the second imaging device 12 may be mounted on the transferringvehicle 91 facing the receiving vehicle 79 (e.g., in FIG. 4) or thereceiving vehicle 79 facing the container 85 (FIG. 5A). In oneembodiment, the second imaging device 12 has a second field of view 177,of the storage portion of the receiving vehicle, where the first fieldof view 77 overlaps at least partially with the second field of view177.

In an alternate embodiment, the second imaging device 12 comprises amonocular imaging device that provides a second image section (e.g.,right image) of stereo image data of a scene or an object, where theimage processing module 18 supports the creation of a stereo image froma combination of the first image section (of the first monocular imagingdevice) and the second image section with reference to the relativeposition and orientation of the first imaging device 10 and the secondimaging device 12.

In step S975, the operational status of the first imaging device isdetermined. If the operational status is not acceptable, then odometryand accelerometer data is used for general alignment of the vehicles(step S978). If the operational status is acceptable, then theoperational status of the second imaging device is determined (stepS976). Thereafter, the operational status of the image processing moduleis determined (step S977). If the operational status of the imageprocessing module is not acceptable, then the system is authorized tooperate in a manual control mode or semi-automated control mode tofacilitate general alignment of the vehicles or determination of therelative position of the spout and the storage portion (step S980). Ifthe operational status of the image processing module is acceptable,then the system is authorized to operate in automated control mode wherethe image processing module processes image data to facilitatedetermination of the relative position of the spout and the storageportion (step S979).

In step S906, an image processing module 18 or a container module 20identifies a container perimeter 81 of the storage portion 93 in thecollected image data (e.g., the first image data, the second image dataor both), where the storage portion 93 has an opening 83 inward from thecontainer perimeter 81 for receipt of the agricultural material. StepS106 may be carried out in accordance with various techniques, which maybe applied alternately or cumulatively. Under a first technique, theimage processing module 18 or container module 20 may employ thefollowing processes or sub-steps: (1) measuring a strength of one ormore edges 181 in the image data (raw and rectified image data); (2)identifying an angle and offset of candidate linear segments in theimage data with respect to an optical axis, reference axis (e.g.,direction of travel of the transferring vehicle), or reference pointindexed to one or more imaging devices 10, 12; and (3) using spatial andangular constraints to eliminate identified candidate linear segmentsthat cannot logically or possibly form part of the identified linearsegments of the container perimeter, where the localizer 111 transformsthe identified linear segments into three dimensional coordinatesrelative to a reference point or reference frame of the receivingvehicle and/or the harvesting vehicle.

Under a second technique, the image processing module 18 or containermodule 20 may receive container reference data, or measurements ofdimensions of the container perimeter 81 or the storage portion 93 ofthe vehicle, to facilitate identification of candidate linear segments,or candidate data points, that qualify as identified linear segments ofthe container perimeter 81.

Under the third technique, the image processing module 18 or containermodule 20 may receive an estimated angle 97 of the storage portion 93relative to the propelling portion 75 of the vehicle to facilitateidentification of candidate linear segments that qualify as identifiedlinear segments of the container perimeter 81.

Under a fourth technique, the image processing module 18 or containermodule 20 provides the received estimated angle 97 of the storageportion 93 relative to the propelling portion 75 of the vehicle.

In step S908, the image processing module 18 or a spout module 22identifies a spout 89 (or spout end 87) of the transferring vehicle(e.g., harvesting vehicle) in the collected image data. The imageprocessing module 18 or the spout module 22 may use various techniques,which may be applied alternately or cumulatively. Under a firsttechnique, the image processing module 18 or the spout module 22identifies candidate pixels in the image data (e.g., rectified or rawimage data) based on expected color and expected texture features of theimage data, where the candidate pixels represent a portion of the spout89 (e.g., combine auger spout) or spout end 87.

Under a second technique, the image processing module 18 or the spoutmodule 22 estimates a relative position, or relative angle, of the spout89 or the spout end 87, to the imaging device based on the classified,identified candidate pixels of a portion of the spout 89.

Under a third technique, the image processing module 18 or the spoutmodule 22 receives an estimated combine spout position, or spout angle,relative to the mounting location, optical axis, reference axis, orreference point of the imaging device (10, 12) based on previousmeasurements to provide constraint data on where the spout 56 can belocated possibly.

Under a fourth technique, the image processing module 18 or spout module22 provides the estimated combine spout position, or estimated spoutangle, to the spout localizer 125.

FIG. 10 is a flow chart of a method for facilitating the unloading ofagricultural material from a vehicle or between a transferring vehicle(91) and a receiving vehicle (79). The method of FIG. 9 begins in stepS900 and may use one or more of the following embodiments of the systems(11, 111, or 311) previously disclosed herein.

The method of FIG. 10 is similar to the method of FIG. 9, except themethod of FIG. 10 replaces step S981 with step S911, as well aseliminated S975, S976, S977, S978, S979, and S980. Like referencenumbers in FIG. 9 and FIG. 10 indicate like elements.

In step S911, the mode controller 225 or image processing module 18determines whether to use an automated control mode of the spout or anoperator-directed manual control mode of the spout 89 based on theacceptability or reliability of a first operational status of a firstlocation-determining receiver 42, a second operational status of asecond location-determining receiver 142, and a third operational statusof the imaging device or devices (10,12).

In one illustrative example, the first operational status is acceptableif the first location determining receiver 42 provides reliable positiondata that meets or exceeds a dilution of precision threshold or anothernavigation satellite reliability measure during a sampling period; thesecond operational status is acceptable if the second locationdetermining receiver 142 provides reliable position data that meets orexceeds a dilution of precision threshold or another navigationsatellite reliability measure (e.g., total equivalent user range error)during a sampling period; and the third operational status is acceptableif the first imaging device 10 provides reliable image data in which thecontainer module 20 or spout module 22 (e.g., or the respective edgedetection modules therein) are capable of any of the following: (1)reliably identifying or resolving one or more edges of spout orcontainer perimeter 81 in the collected image data during a samplingtime period, or (2) reliably identifying on a time percentage basis(e.g., at least 99.99% of the time) one or more reference objects (e.g.,a reference pattern or reference image on the spout or receivingvehicle) or objects in the image data.

Step S911 may be executed in accordance with one or more procedures thatmay be applied separately or cumulatively. Under a first procedure, ifthe first, second and third operational statuses are not acceptable orreliable, the system (11, 111, 311) or mode controller 225 reverts tomanual mode control for control of the spout 89 and container 85alignment and the spatial alignment of the transferring vehicle 91 andthe receiving vehicle 79 during unloading of agricultural material fromthe transferring vehicle 91 to the receiving vehicle 79.

Under a second procedure, if the first operational status, the secondoperational status and the third operational status are acceptable orindicative of properly functioning, the system (11, 111, 311) is capableof operating in an automated mode (e.g., fully automated mode) and themode controller 225 permits the use of the first location determiningreceiver 42, the second location determining receiver 142 and theimaging device 10 for one or more of the following: (1) to track oralign relative position of the receiving vehicle 79 and the transferringvehicle 91 for control of the spout 89 and container 85 spatialalignment during unloading, and (2) to track or align the spatialalignment of the transferring vehicle 91 and the receiving vehicle 79during unloading of agricultural material from the transferring vehicle91 to the receiving vehicle 79.

Under a third procedure, if the first operational status, the secondoperational status and the third operational status are acceptable orindicative of properly functioning, the system (11, 111, 311) or modecontroller 225 is capable of operating in an automated mode (e.g., fullyautomated mode) and determines if there is a storage portion 93 orcontainer 85 (e.g., cart) properly positioned beneath the spout end 87of the spout 89 so that no material (e.g., grain) will be spilled whenthe material begins to flow from the spout end 87 to the receivingvehicle. In the automated control mode, if a storage portion 93 isdetected and properly positioned with respect to the transferringvehicle 91 and the spout 89, the system (11, 111, 311), imaging module18 or vehicle controller 46 commands the combine to turn on theunloading auger or auger drive 47.

Under a fourth procedure, if the first operational status and the secondoperation status are acceptable and if the third operational status isunacceptable, the mode controller 225 permits the use of the firstlocation determining receiver 42 and the second location determiningreceiver 142 to track or align the relative position of the receivingvehicle 79 and the transferring vehicle 91 during unloading. However,the mode controller 225 may prohibit use of the imaging device (10,12)to track or align the relative align relative position of the receivingvehicle 79 and the transferring vehicle 91 for control of the spout 89and container 85 spatial alignment during unloading. Accordingly, whereoperating under the fourth procedure, the mode may be referred to aspartially automated or partially manual mode because automated aspect islimited to the first location determining receiver 42 and the secondlocation determining receiver 142 to track or align the relativeposition of the receiving vehicle 79 and the transferring vehicle 91during unloading, subject to manual adjustment of the operator of thetransferring vehicle 91 within certain tolerance to align manually thespout 89 and the container 85 of the receiving vehicle 79 for unloading.The tolerance is limited to a spatial separation to prevent or minimizethe risk of collision between the transferring vehicle 91 and thereceiving vehicle 79 based on the respective dimensions, wheel base,turning radius, speeds, velocities and headings of the transferringvehicle 91 and the receiving vehicle 79.

Under the fifth procedure, the mode controller 225 permits (e.g.,temporarily permits) the image processing system (11, 111, 311), themode controller 225 and the steering controller 32 to use collectedimage data from the first imaging device 10, odometry data from theodometry sensor 440, and inertial data from the inertial sensor 442 ifthe first operational status and the second operational status areunacceptable and if the third operational status is acceptable.Accordingly, under the fifth procedure the mode controller 225authorizes the system (11, 111, 311) to operate in an automated mode(for a limited time period measured from a last available position datafrom one or more location determining receivers) or a partiallyautomated mode in which the user can override or adjust the spatialseparation between the transferring vehicle 91 and the receiving vehicle79 or the spatial alignment between the spout 89 and the container 85 ofthe receiving vehicle 79. For example, the steering controller 32operates the system (11, 111, 311) in a partially automated mode inwhich an operator supervises the steering system 30 of the transferringvehicle 91 and the receiving vehicle 79; where the operator can overridethe automated steering control of the steering controller 32 by themaster controller 59 based on the odometry data and inertial data; afterthe limited time period expires, the mode controller 225 only authorizesoperation in the manual mode, unless the first operational status andthe second operational status become acceptable or reliable prior toexpiration of the limited time period.

Under a sixth procedure, the storage portion 93 of the receiving vehicle79 begins to become full, the system (11, 111, 311), propulsioncontroller 40, braking controller 36, vehicle controller 46, and augerrotation system 16 may use the profile of the surface the agriculturalmaterial to execute the operator-selected fill strategy (e.g.,back-to-front, front-to-back, or another fill strategy).

To execute the fill strategy in the automated control mode, the system(11, 111, 311) can unload the agricultural material into particularareas of the container 85 that contain less agricultural material topromote even filling and distribution of agricultural material in thecontainer 85. In the automated control mode, the system (11, 111, 311)can adjust the unloading alignment between the transferring vehicle 91and the receiving vehicle 79 by one or more of the following procedures:(1) command the spout 89 to rotate or change its spout angle withrespect to the transferring vehicle 91 or a vertical plane associatedwith the transferring vehicle 91 that intercepts at least one rotationalaxis of the spout 89; (2) command the unloading auger or auger drive 47to rotate to move agricultural material from the storage in thetransferring vehicle to the storage portion 93 of the receiving vehicle;(3) command changes to ground speed, velocity, acceleration or headingof the transferring vehicle 91, the receiving vehicle 79, or both; (4)command the operator of the vehicle or vehicles (79, 91) to manuallyadjust ground speed, velocity, acceleration or heading of thetransferring vehicle 91, the receiving vehicle 79, or both, or (5)command one or more controllers (32, 36, 38) to command the transferringvehicle 91, the receiving vehicle 79 or both to change its or theirrelative position. When the entire container 85 or storage portion 93 isfilled to capacity or the level selected by the operator, the vehiclecontroller 46 or auger rotation system 16 turns off the unloading augeror the auger drive 47.

Under a seventh procedure in a manual control mode, the operatorinteracts with the user interface 44 of the system (11, 111, 311) toadjust the unloading alignment between the transferring vehicle and thereceiving vehicle by any of the following: (1) the operator's changingground speed, velocity, acceleration or heading of the transferringvehicle 91, the receiving vehicle 79, or both, or (2) the operator canadjust manually the spout angle based on imaged displayed to an operatoron the user interface 44 from an imaging device (10, 12). Further, inthe manual control mode, the operator may turn or off the unloadingauger or auger drive.

The method of FIG. 11 is similar to the method of FIG. 9, except themethod of FIG. 11 further comprises step S916 and S918. Like referencenumbers in FIG. 9 and FIG. 11 indicate like steps or procedures.

In step S912, the image processing module 18 or the alignment module 24determines the relative position of the spout 89, or the spout end 87,and the container perimeter 81 and for generating command data to thepropelled portion to steer the storage portion 93 in cooperativealignment such that the spout 89 (or spout end 87) is aligned with acentral zone 83 of the container perimeter 81. The image processingmodule 18 may use, retrieve or access previously stored data, such asdimensional parameters related to the receiving vehicle, the dimensionalparameters comprising a distance between a trailer hitch and front wheelrotational axis of the storage portion 93. Such dimensional parametersmay be entered via a user interface 44 coupled to the vehicle data bus31 or the image processing module 18, for example.

To execute step S912, the imaging processing module 18 may use firstlocation data of a first location determining receiver 42 on thetransferring vehicle and second location data of a second locationdetermining receiver 142 on the receiving vehicle to determine one ormore of the following: (1) a relative position of the first imagingdevice 10 and the second imaging device 12, where the first imagingdevice 10 and the second imaging device 12 are on different vehicles orcan experience relative movement with respect to each other, (2) arelative spatial separation between fixed reference points (e.g.,antennas of the location determining receivers (42, 142)) on thereceiving and transferring vehicles, (3) relative alignment between thespout and the container perimeter, (4) spatial separation and anglebetween reference points on the transferring vehicle and receivingvehicle to achieve relative alignment or target spatial offset betweenthe spout 89 and the container perimeter 81 to support reliableunloading of agricultural material into the container 85 of thereceiving vehicle from the spout. If the first imaging device 10 and thesecond imaging device 12 are mounted to fixed portions of the samevehicle, the relative spatial alignment between the first imaging device10 and the second imaging device 12 may be fixed.

In step S914, in a first configuration, the controller (59 or 159) orthe steering controller 32 steers the receiving vehicle in accordancewith the cooperative alignment. In a second configuration, the vehiclecontroller or the steering controller 32 may steer the transferringvehicle in accordance with the cooperative alignment. In a thirdconfiguration, the vehicle controller (59 or 159) or steeringcontrollers 32 of both the transferring vehicle 91 and the receivingvehicle 79 steer both vehicles in accordance with the cooperativealignment, or maintenance of a target spatial offset suitable forunloading or transfer of the material between the vehicles. In a fourthconfiguration, the actuator 116 (e.g., a servo-motor, electric motor,linear motor and linear-to-rotational gear assembly, orelectro-hydraulic device) controls the spout angle of the spout 89, orthe spout end 87, with respect to the direct of travel or anotherreference axis of the transferring vehicle in response to alignmentmodule 24 or the image processing module 18 (e.g., smart unloadingcontroller).

Although the imaging devices 10, 12 are susceptible to transitorysunlight, shading, dust, reflections or other lighting conditions thatcan temporarily disrupt proper operation of the imaging devices in anagricultural environment; the system and methods disclosed in thisdocument are well suited for reducing or eliminating the deleteriouseffects associated with material changes in ambient light conditions.Accordingly, the system and methods disclosed in this document supportaccurate guidance and alignment of the spout and the counter even whereambient light conditions fluctuate.

The method and system is well suited for enhancing the efficiency ofunloading of a transferring vehicle (e.g., combine) to a receivingvehicle (e.g., tractor pulling a grain cart) by facilitating thevelocity or speed matching of the vehicles via position data fromlocation determining receivers, where fine tuning of the alignment ofthe spout end and the container perimeter is supported by image datafrom one or more imaging devices. In the absence of the method andsystem disclosed herein, the operator of the receiving vehicle tends toset a constant speed that is below the optimal speed for harvesting toavoid spilling agricultural material on the ground and missing thecontainer of the receiving vehicle. Accordingly, the method and systemis well suited for reducing the time to harvest a field and to collectthe grain than otherwise possible.

While the disclosure has been described in detail and with reference tospecific embodiments thereof, it will be apparent to one skilled in theart that various changes and modifications can be made therein withoutdeparting from the spirit and scope of the embodiments. Thus, it isintended that the present disclosure cover the modifications andvariations of this disclosure provided they come within the scope of theappended claims and their equivalents.

1. A method for facilitating the transfer of material from atransferring vehicle having a material distribution end to a receivingvehicle having a bin to the store transferred material, the methodcomprising the steps of: a. identifying and locating the bin; b.detecting a representation of the fill level or volumetric distributionof the material in the bin; c. aligning the material distribution endover a current target area of the bin requiring the material; d.determining subsequent target areas of the bin that require materialbased on the representation of the fill level or volumetric distributionof the material in the bin and a desired fill pattern to fill the bin;e. transferring the material from the transferring vehicle to thecurrent target area of the bin of the receiving vehicle; f. detectingwhen the current target area of the bin is filled with the material; andg. repeating steps c-f until the subsequent target areas of the bin arefilled per the desired fill pattern.
 2. The method according to claim 1,wherein the representation of the fill level or volumetric distributionof the material in the bin is one-dimensional.
 3. The method accordingto claim 1, wherein the representation of the fill level or volumetricdistribution of the material in the bin is two-dimensional.
 4. Themethod according to claim 1, wherein the representation of the filllevel or volumetric distribution of the material in the bin isthree-dimensional.
 5. The method according to claim 1, wherein the stepof detecting the representation of the fill level or volumetricdistribution of the material in the bin further comprises the steps of:receiving data from a distributed fill state sensor; generating therepresentation of the fill level or volumetric distribution of thematerial in the bin based on the distributed fill state sensor.
 6. Themethod according to claim 1, wherein the step of detecting therepresentation of the fill level or volumetric distribution of thematerial in the bin further comprises the steps of: receiving rectifiedimage data from a single imaging device; generating disparity image databased on the rectified image data; and generating the representation ofthe fill level or volumetric distribution of the material in the binusing range data based on the disparity image data.
 7. The methodaccording to claim 1, wherein the step of determining subsequent targetareas of the bin that require material based on the representation ofthe fill level or volumetric distribution of the material in the bin anda desired fill pattern to fill the bin comprises the steps of:developing a target zone matrix of the bin, wherein each target zone ofthe matrix is identified by a pre-established set of coordinates relatedto the bin; identifying target zones of the matrix that are filled andnot filled based on the representation of the fill level or volumetricdistribution of the material in the bin; and determining thepre-established set of coordinates of a subsequent target zone to befilled and the pre-established set of coordinates of the current targetzone over which the material distribution end is positioned.
 8. Themethod according to claim 7, wherein the step of aligning the materialdistribution end over a current target area of the bin to be filledcomprises the step of actuating a control mechanism to move the materialdistribution end over the pre-established set of coordinates of thesubsequent target zone to be filled.
 9. The method according to claim 7,wherein the step of aligning the material distribution end over acurrent target area of the bin to be filled comprises the step ofchanging a lateral offset between the receiving vehicle and thetransferring vehicle to move the material distribution end from over thecurrent target zone to over the subsequent target zone to be filled. 10.The method according to claim 9, wherein the step of changing a lateraloffset between the receiving vehicle and the transferring vehiclecomprises the step of: establishing a current lateral offset between thereceiving vehicle and the transferring vehicle; converting thepre-established set of coordinates between the current target zone andthe subsequent zone to be filled into a change in the current lateraloffset to form a new lateral offset; transmitting the new lateral offsetto a steering controller of the receiving vehicle; and steering thereceiving vehicle to create the new lateral offset between the receivingvehicle and the transferring vehicle, whereby the material distributionend is aligned the over the subsequent target zone to be filled.
 11. Themethod according to claim 7, wherein the step of aligning the materialdistribution end over a current target area of the bin to be filledcomprises the step of changing a fore/aft offset between the receivingvehicle and the transferring vehicle to move the material distributionend from over the current target zone to over the subsequent target zoneto be filled.
 12. The method according to claim 11, wherein the step ofchanging a fore/aft offset between the receiving vehicle and thetransferring vehicle comprises the step of: establishing an initialfore/aft offset between the receiving vehicle and the transferringvehicle; converting the pre-established set of coordinates between thecurrent target zone and the subsequent target zone to be filled into achange in the current fore/aft offset to form a new fore/aft offset;transmitting the new fore/aft offset to a propulsion controller of thereceiving vehicle, or a braking controller of the receiving vehicle; andaccelerating or braking the receiving vehicle to create the new fore/aftoffset between the receiving vehicle and the transferring vehicle,whereby the material distribution end is aligned the over the subsequenttarget zone to be filled.
 13. The method according to claim 1, whereinthe step of detecting when the current target area of the bin is filledwith the material comprises the step of detecting a representation ofthe fill level or volumetric distribution of the material in the currenttarget area of the bin.
 14. The method according to claim 13, whereinthe step of detecting the representation of the fill level or volumetricdistribution of the material in the current target area of the binfurther comprises the steps of: receiving data from a distributed fillstate sensor; and generating the representation of the fill level orvolumetric distribution of the material in the current target area ofthe bin based on the distributed fill state sensor data.
 15. The methodaccording to claim 13, wherein the step of detecting the representationof the fill level or volumetric distribution of the material in thecurrent target area of the bin further comprises the steps of: receivingrectified image data from a single imaging device; generating disparityimage data based on the rectified image data; and generating therepresentation of the fill level or volumetric distribution of thematerial in the bin using range data based on the disparity image data.16. The method according to claim 1, wherein the material isagricultural material.
 17. The method according to claim 1, wherein thematerial is a mineral material.
 18. The method according to claim 6,wherein the step of receiving rectified image data from a single imagingdevice further comprises receiving a rectified image data from animaging device on the receiving vehicle, wherein there are no imagingdevices on the transferring vehicle.
 19. The method according to claim15, wherein the step of receiving rectified image data from a singleimaging device further comprises receiving a rectified image data froman imaging device on the receiving vehicle, wherein there are no imagingdevices on the transferring vehicle.